Palanthir BV

Charging electric vehicle without the cloud

2025-11-24T00:00:00+01:00

My wife and I both drive electric vehicles (EV) for years now. More than 6 years ago we got our first EV for my company. To be able to charge it at home and be able to declare the charging sessions to my company we got a charger from EVBox including a corresponding subscription which handled the invoicing of the charging sessions to my company. The cloud software from EVBox handled the invoicing of each charging session to my company, and paid out the money to my private bank account since the electricity loading into the car comes from our private home electrical connection to the grid.

This worked fine for a couple of years.

We had a business line charger with a mobile 3G connection. 2 years ago the mobile network providers in the Netherlands decided to phase out 3G. I never received the email from EVBox that our charger needed to be upgraded to function after 2025. Upgrade kits were sold until July 2024, costing about € 400, and by accident I found out about this necessary upgrade months after that.

Then in the fall of 2024 EVBox announced to completely stop with their charger business. From December 1st 2025, our charger would function only as a regular home charger without any registration of charging sessions whatsoever. There are other cloud payment providers that you can switch to to keep the full functionality, but I received an email earlier this year that this was not the case for our charger. It seems our charger did not have the firmware to be able to switch to another provider.

Bummer.

So, basically we needed a new charger. We have used the payment services of EVBox for over 6 years now and we are stuck with an e-waste charger, forced to buy a new charger and everybody is saying 'just switch to another provider'. I know almost everything works now only when connected to the cloud, but in this case I am tired of paying for a subscription that only sends an invoice every month and handles the bank transactions.

Learning about OCPP

So I started to shop for a new car charger that I could manage myself without any cloud service. In the car charging world, there is a standard for exactly this purpose: OCPP, Open Charge Point Protocol. Most chargers you can buy nowadays have this, but it is tricky. There are a lot of car chargers available that support OCPP, but only via the cloud of their manufacturer. If I want to be able to manage my new car charger myself, I need a device that can be setup to use OCPP directly.

So I went shopping and found the Zaptec Go. In the specification it says clearly "OCPP 1.6J cloud-to-cloud or native on-device". Exactly what I needed, the smallest size I could find, and the cheapest one, so I ordered it.

To manage a car charger that can be managed by OCPP, I also needed some back-end software and found that in the community open source software Steve.
With this software we can manage the car charger, manage which NFC tags are authorized to start a charging session, and it keeps track of the charging sessions.

Installing the new car charger

So I ordered the Zaptec Go and a week ago I had everything prepared to make the switch.

It took me 2,5 hours to disassemble the EVBox BusinessLine charger, that (big) device is really not made for that. In the end I used a jigsaw to be able to remove the cover of it.

Putting the Zaptec Go on the wall was done in 30 minutes. Of course I had the benefit that the cable to the charger was already present, so it was just putting 4 screws in the wall and connect the wires.

Configuration of the charger

Nice, now we can start setting up the new charger! I already suspected this would not be a simple task, and I was right. Setting the charger up with a bluetooth connection in their app was simple. After that, the charger was connected to my guest WiFi network which I also use for devices like this that do not belong on our own network. Zaptec even offers a more extensive web portal to manage your charger from there.

Of course, by default the charger connects to the cloud of the manufacturer, so although I could see the charger would work, I could not start a charging session yet because I had no RFID/NFC tag authorized yet. You can start a charging session directly from the mobile app also, but then I had to find out how to export charging sessions from the Zaptec portal, and the main goal was to implement a solution that:

Stays local and works, even when our internet connection is down
Keeps local track of our charging sessions, so we can
Automatically send an invoice each month for the charging sessions.

So I started reading the documentation from Zaptec. It mentioned something about user permissions which I could not find when I was logged into their portal, and basically it describes to enable OCPP as follows:

I had the first 3 options, but not the last one, exactly the one I needed. My own install of the Steve software was not publicly accessible from the Internet (obviously for security reasons), so I could not get it to work.

The URL of my Steve installation was https://steve.int.palanthir.nl, notice the int.palanthir.nl domain name, which is not accessible from the Internet.

By now it was Saturday evening about 11 PM...

I realized I needed support from Zaptec, so I sent a support request to them through their website, not expecting much of it. Especially during the weekend I do not expect support from a manufacturer to be available and respond to me.

After this, I spent some hours reconfiguring my network and opening ports in my firewall to have the Steve software available from the Internet in a somewhat secure way. I got a working setup with their 'OCPP 1.6J Cloud' authentication type and the following URL of my Steve OCPP management software:

wss://steve.palanthir.nl/steve/websocket/CentralSystemService/

Although I now had a working setup, it was still using the cloud, not an option for me. With all the connections working, I could now write a script to retrieve the charging sessions from the Steve software and create an invoice in my invoicing software.

Zaptec support

On Monday afternoon I received an email from Zaptec support that OCPP was not implemented in the Netherlands yet, but they were trying to find out how to help me. It took about 4 days of emailing back and forth until I had the option to configure the direct OCPP variant of their authentication type named "OCPP 1.6J". This profile was needed to get it working.

But it didn't. Like above, I set the OCPP websocket URL to this:

wss://steve.int.palanthir.nl/steve/websocket/CentralSystemService/

I now suspected some DNS problem or whatever, so I started digging in the logs of the Steve software to find out what was going wrong. By default, Steve is configured with the INFO log level, which did not give me any clue. So I reconfigured that to the DEBUG log level, which gives a lot of debug logging. After some digging, I found out the authorization of the car charger to Steve was not working because the URL the car charger now was using to authenticate itself was

wss://steve.int.palanthir.nl/steve/websocket/CentralSystemService//<device serial number>

Notice the 2 slashed in the URL? That was the culprit. When I set the Websocket URL in the Zaptec portal to

wss://steve.int.palanthir.nl/steve/websocket/CentralSystemService

without the trailing slash character, it worked within seconds!

I tested this configuration also by disconnecting the Internet connection at home, and the car charger kept working and authenticating by using the NFC cards I authorized for charging.

Conclusion

I understand that for a manufacturer the most easy way to configure their devices is to have a cloud service available for that. This makes the firmware in the devices smaller, and their cloud software is easier to maintain and up to date. I am very grateful that manufacturers like Zaptec still exist to make their devices work without an Internet connection.

And I am even more grateful that I now have a setup that does not require a payment provider with a subscription to handle 12 invoices a year and pay for it.

The downsides:

The Steve software does not look as modern as mobile apps or a modern web portal looks like.
When I need to change settings in the car charger, I need to do it through the Steve software, the mobile app from Zaptec does not show my car charger anymore.

The advantages:

No subscription needed any more. I have a monthly Python script now that takes care of the invoicing.
If Zaptec in the future decides not to support my charger any more, it keeps working.
Even if our home Internet connection goes down, I still can start and stop charging sessions.
We now have a solution which is extendable for the future, for example when we will have a home battery and do even more smart charging than the Zaptec portal can provide.

Lots of thanks to the support team at Zaptec for giving me access to the correct authentication profile.

Time ranges in postgresql with GIST indexes

2025-09-16T00:00:00+02:00

At ColoClue we need a database which registers the resources members of the organization use. For example, we need to register which switch port on a network switch is being used by a member. This also counts for power bar ports, rackspace in a rack, access cards, cross connect links in patch panel, and more.
Years ago we had often conflicts representing the contents of this database compared to the real world situation. ColoClue often suffered situations in which there were conflicting records in the database with overlapping time ranges for a specific switch port or power bar port. This cannot happen in the real world: in one switch port can physically go only one cable. These overlapping records meant that someone had to go to the data centre to check what the real situation was at that moment. This is far from ideal.

Also, because invoicing the use of resources happens only once a month, ColoClue has to keep a history of mutations to the database to correctly invoice the use of resources. For example, when someone starts to use a new power bar port on the 2nd of the month and stops using that before the invoices for that month are created a month later, ColoClue still has to charge money for it. So, keeping track of history is needed in the database.

This is like the classic book lending problem for libraries, which is a more generic example. When someone lends a book from a library, the library needs to register which person has a book. And not only that, only one person can lend that specific book for a specific period of time. And, also a library need to keep track of these interactions with the book.

This can be done in a relational database with a table design like this:

CREATE TABLE book_lending (
  id serial4 NOT NULL
, person_id int4 NOT NULL
, book_id int4 NOT NULL
, start_datetime timestamptz NOT NULL
, end_datetime timestamptz NULL
);

With this table the combination with person and book can be registered. For each lending action, the start date and time are being registered, and when the book is returned, that data and time are registered as end date. When the end_datetime is empty, this means the person is still lending the book.

This is a simple table design for registering the lending process. However, what it does not do is preventing the software that uses this table to insert overlapping time ranges. So it is possible to have records in the database that overlap the ranges used. The obvious solution for this is to have constraints that prevent this. This general problem occurs in a lot of software development projects.

Almost every database has the possibility to add check constraints to a table to prevent this. However, these constraints can often become quite complex for situations like this, especially when taking into account the removal_dt column for soft deletes described in my article about tracking history in a database. Also, they are difficult to debug. That is why with a lot of software projects the constraint checking for this behavior is written in the middleware layer of the software, especially when writing software that has to be database independent.

At ColoClue, we however had the situation that the database was accessed from different sources. Regardless of that, the best solution is to have the check constraints implemented directly on the database itself. This assures time range overlapping is not possible at all times from regardless from which source the database is being modified.

PostGreSQL has a very elegant solution for this: the tstzrange column type in combination with the GiST indexes functionality.

GiST indexes are an extension to PostGreSQL, available for almost any PostGreSQL instance. GiST indexes are often used in databases using geographical databases. They can be enabled by executing the following once on the current database schema.

CREATE extension btree_gist;

Using this, the table design looks like this:

CREATE TABLE book_lending (
  id serial4 NOT NULL
, person_id int4 NOT NULL
, book_id int4 NOT NULL
, during tstzrange NOT NULL,
, removal_dt timestamptz NULL,
);

ALTER TABLE book_lending
ADD CONSTRAINT gist_book_lending EXCLUDE
USING GIST (
  book_id WITH=
, during WITH &&)
WHERE (removal_dt IS NULL)
;

The constraint added describes to check for it only when the removal_dt column is empty. We do not need to check for soft deleted data, making the index faster. Further, the operators used for checking the constraint determine how each column is checked. For the book_id column this is simple, it has to check (the = operator) for non-conflicting books. The && basically checks for overlapping during time ranges.

The tstzrange data type is specific for PostGreSQL. It is handled a bit different from standard data type while querying and inserting data. Because it is a time range, it uses parentheses and square brackets to indicate the start- and end times of the range. The parentheses or brackets indicate whether the lower and upper bounds are exclusive or inclusive. Inserting a person lending a book will look this this:

INSERT INTO book_lending (person_id, book_id, during)
VALUES (1, 134, tstzrange('2025-09-01 12:00:00', null, '[)'));

id	person_id	book_id	during	removal_dt
1	1	134	["2025-09-01 12:00:00+00",)

Now, if someone else tries to lend the book an hour later, and error occurs:

INSERT INTO book_lending (person_id, book_id, during)
VALUES (2, 134, tstzrange('2025-09-01 13:00:00', null, '[)'));

SQL Error [23P01]: ERROR: conflicting key value violates exclusion constraint "gist_book_lending"
  Detail: Key (book_id, during)=(134, ["2025-09-01 13:00:00+00",)) conflicts with existing key (book_id, during)=(134, ["2025-09-01 12:00:00+00",)).

If the first person returns the book, we handle that like this using the same lending date and time the person did lend the book by using lower(during) which takes the lower part of the time range:

UPDATE book_lending
SET during = tstzrange(lower(during), '2025-09-16 10:00:00', '[)')
WHERE id = 1;

id	person_id	book_id	during	removal_dt
1	1	134	["2025-09-01 12:00:00+00","2025-09-16 10:00:00+00")

If now the second persons lends the book, the constraint still goes of if there is still an overlap in the time range. So this does not work:

INSERT INTO book_lending (person_id, book_id, during)
VALUES (2, 134, tstzrange('2025-09-16 08:59:59', NULL, '[)'));

SQL Error [23P01]: ERROR: conflicting key value violates exclusion constraint "gist_book_lending"
  Detail: Key (book_id, during)=(134, ["2025-09-16 09:59:59+00",)) conflicts with existing key (book_id, during)=(134, ["2025-09-01 12:00:00+00","2025-09-16 10:00:00+00")).

But this works because the top of the time range is set with until instead of until and including the higher time stamp:

INSERT INTO book_lending (person_id, book_id, during)
VALUES (2, 134, tstzrange('2025-09-16 10:00:00', NULL, '[)'));

These checks are exactly what is necessary for the book lending constraints problem, solved with only a few SQL commands.

Drawbacks

GiST indexes are a little bit slower than standard B-tree indexes. I did not do a benchmark compared to writing the check constraints myself, but I never had any performance problems with it.
Getting the start - and end timestamps of the values of the during column can be a little bit more complicated, especially when using an ORM. This can be easily solved by creating a view that separates the lower and upper time range values:

CREATE VIEW vw_book_lending AS
SELECT id, person_id, book_id, LOWER(during) start_timestamp, UPPER(during) end_timestamp, removal_dt
FROM book_lending;

SELECT * FROM vw_book_lending;

id	person_id	book_id	start_timestamp	end_timestamp	removal_dt
1	1	134	2025-09-01 12:00:00.000 +0000	2025-09-16 10:00:00.000 +0000
6	2	134	2025-09-16 10:00:00.000 +0000

Conclusion

Advanced PostgreSQL types are underrated, they provide strong capabilities.
When using GiST and exclusion constraint, you can enforce complex logic at the database level. The error messages returned even show the conflicting record so you do not have to go and search for it in the table contents.
By enforcing the constraint on the database level, it does not matter anymore from which source code the database is accessed. Data integrity is always enforced.

Concerns about computer programming with AI

2025-09-02T00:00:00+02:00

A few months ago I decided to publish more content on this web site. When I published my first articles, the first response I received about those was "so refreshing to read an actual personally written article again, instead of generated by AI". I guess that is reality nowadays.

In 2023 Microsoft released CoPilot. First as general chat bot and later Microsoft integrated the functionality into development environments like Visual Studio (Code). It started as an AI code assistant suggesting improvements of the code a developer is writing, this later improved into Vibe coding products that can do a lot of programming for you.
Since then CoPilot, Cursor, Claude and other Vibe coding tools have been booming.

Almost every review I read about coding using AI, often referred to as Vibe coding, started with something like "I am not against Vibe coding" and then the pros and cons of using these tools.

Of course I have played with some of the vibe coding tools to see how it performs, in 2024 my experience was "promising but quite not there yet". I was unimpressed. However, things are moving fast in this space and earlier this year I tried again to make a small pilot project. I can see the results are becoming much better nowadays. People without much understanding of how to code can get results that at least deliver code thats does what they want.

After reading this article about someone stopped using AI code editors, my own reservations about using vibe coding were more confirmed.

Intellectual property

Almost every commercial company nowadays has some form of IT implemented in their company. This can be using only standard software, and often with (at least partly) custom programming and databases. This code and the content of these databases are the intellectual property of the business.
Often that is their unique value compared to competitors. For example, if a company uses some form of customer relation management software, the contents of the database of that software, all their clients and information about their clients is stored there.

This is very valuable information. I cannot imagine any company wants to give that information away and even pay for it.

In my professional environment I could not use vibe coding tools yet: when using AI assisted coding like like CoPilot or Claude, (snippets of) the code you write are sent to the AI provider to train their AI model(s). When I write code for my clients, usually the intellectual property rights are owned by the client. So when Microsoft started to push their CoPilot product in almost every product, I had to be sure no client code was going to leak into the AI training of CoPilot or whatever other AI assisted coding tool.
I am not in the position to make this decision for the companies I consult.

In the beginning of 2024 it was easy to do by disabling CoPilot in Microsoft Visual Code, but in the beginning of this year Microsoft started to push CoPilot so much that I could not trust anymore that no code I was writing for my clients was leaking to Microsoft, so I switched to VSCodium which is the same programming environment without all the Microsoft telemetry.

Not only do the vibe coding tools ask for your code, they now also want to have access to the database the code has to work with. I can understand that in a way, because most software only works perfectly in cooperation with their corresponding databases.
In a perfect world all companies and everybody only use mockup test data within their databases and keep production data in a complete separate environment of course, but be real, this often does not happen. It is a huge risk.

In the past a lot of programmers used tutorials to implement several services like AWS S3. Every tutorial says "for this demo we do not implement credentials of this service, but you should do it for your production project" and of course there were a lot of data leaks of data stored in AWS S3 buckets that should have never happened.
These security leaks still happen.

Consider the following scenario: you have an existing project and want to use to AI to make improvements on it. To accomplish this, you have to give the AI access to your code and database to make it understand on what to make improvements. Even if you use a database with mockup data, the AI still has access to your code and trains itself from it.
And what happens when there is a bug in the production environment with production data not covered in your mockup database? We are all humans, it could be faster to give the AI access to that database in the production environment too. And by a simple small configuration error you gave the AI provider all of your data.

The cause of the data breach is then basically a user error, not the AI, but I think we just have to wait until such data breaches happen. And if your data is in the hands of the AI company, good luck with that.

Creativity

Large Language Models (LLM) are very impressively good in recognizing patterns. It is trained on A LOT of data and based on that it can predict what mostly will be the next word it has present to you. Every LLM has a lot of parameters where it can be 'polished' to behave in a certain way. They also have a 'creativity' parameter that makes sure it generates a different answer each time a more complex question is asked. This way they do not give the exact same answer for every question to every user. (It works somewhat different of course, but I hope you get the picture).

You can ask a LLM to write a story for you, or just a complete book and it can be very appealing. But somehow all those created texts often feel somewhat 'off'. The first responses I got on my articles mentioned on the top of this article are an expression of that. There is even a whole science evolving around that.

If I ask a LLM to setup a Python FastAPI project for me, it creates a file structure that works. Very handy, I get a speed jump starting a new project and that is a benefit for a programmer. But if a whole program or web site is generated by AI you do not get the creativity needed for that program, you get the average code the LLM was trained on , usually by getting the code from Github. But all that publicly available code it was trained on, also has A LOT of bad code the LLM is also trained on.

A LLM basically works like the principe of wisdom of the crowd that generally leads to good results. Never the best result, but good enough.
Is that a bad idea? Probably not, if a company wants to hire the best programmer, it will probably also decide to hire one that is 'good enough' because the best is probably the most expensive.

By using vibe coding you by definition accept you only get results that are at best 'good enough'.

A human programmer will try to deliver the best in the time he has to accomplish a task. It will probably contain bugs, it will be a different from the average generated by an AI tool, but the programmer will have thought of it, and probably has put one or two really smart decisions in the code, and probably kept other (future) requirements in mind to create it.
It will probably have more use for the company he is writing it for, and thus have higher quality. Basically, he puts his creativity into it.

If a writer of a novel writes a book, he wants his book to be good. Not because is also sells better, but he/she will be proud of it. If you tweak a LLM to write books that sell the best, they will turn out more and more of the same.

Business case of vibe coding tools

AI is a hype at the moment. In the beginning of the internet, a lot of Internet Service Providers (ISP) provided free internet access in which the users became the product. The internet was still small at that time, investments in the networks were high, and ISP's needed money to grow. So they gave the access to the internet away for free to leverage maximum growth to get as much users as possible.
The ISP's sold the customer profile data of their customers to advertisement companies which then could build more specific profiles of all those new online users to target ads more efficiently. This made the internet profitable very quickly.
After this initial hype was over, free ISP's basically disappeared and we now all pay for internet access to view ads served by those advertisement companies that have a more and more detailed profile of each person online every day.

Another example is the cloud computing business. When the large cloud providers started, they only had a minimal viable product they sold for cheap, even for free. AWS, Google and Microsoft still offer free services, just enough to try something out, but when you really want to use services they offer, you pay more for it now than hosting it yourself. Nowadays a lot of businesses fully rely on the cloud, resulting in vendor lock-in and almost no way back.

AI companies use the same strategy.

In the beginning, using the LLM's was free, as the use of ChatGPT sky rocketed, more advanced functionality became paid very quickly. They invest billions of dollars in it, but it not profitable yet probably for all AI companies. AI looks exactly like the Dot-Com Bubble. What they now offer is a low price for the services they offer.

All the AI companies promise their large language models (LLM) will eventually lead to a form of Artificial General Intelligence (AGI), but it is becoming clearer and clearer these LLM's are hitting a wall to achieve that.
All the investments the AI companies now make to achieve this, must become profitable in the near future, so I am afraid those prices will become much higher in the future.

As with Vibe coding, AI companies want you to believe coding is not necessary any more.

In the same time as AI became more and more a hype, a lot of programmers were losing their jobs. Not only because the AI was becoming better and better, but by a large part because those engineers were becoming too expensive for the big tech companies and AI was a better excuse to blame. Vibe coding tools have also a free tier, but if you really want to accomplish thing you pay by using tokens you have to buy. This is a never ending story.

This means less jobs for computer programmers, and the AI companies happily jump into the gap, because companies still need programming roles and AI is now cheap.

But what if there are no computer programming roles left? What if AI becomes so expensive it will cost more than a programmer costed in 2024? The answer you usually get is that with new technology there is a shift of jobs in the long term. There are now people needed in new roles like AI prompting roles?

I have doubts in this case. Vibe coding does not seem the holy grail to me and the AI companies want to replace too much jobs, not only computer programmers. The costs of a generic business will rise in the long term only benefiting the few AI big tech companies. Of course I cannot predict the future, but what happened with the large cloud providers and their vendor lock-in makes me nervous.

My advice for now

Be very careful of giving away your intellectual property to AI companies, especially the contents of your databases.
Make use of AI as a tool to increase productivity at non important parts of your company at least for now, do not rely on vibe coding only yet.

Python compile problem on Linux as Samba user

2025-06-16T00:00:00+02:00

Last week I had a head banging problem that had me pulling my hair. Although I do not do it often, I needed to install a specific version of Python on my laptop. Usually I use asdf or py-env to accomplish this. Both utilities are package managers to install a specific version of a package for local use. They do that by compiling the Python version locally and install it in a directory in your home directory to avoid the versions installed system-wide. This makes switching between Python versions for different projects easier.
So I just typed in the following command:

 $ asdf install 3.13.2
remote: Enumerating objects: 8, done.
remote: Counting objects: 100% (8/8), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 8 (delta 0), reused 1 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (8/8), 26.01 KiB | 2.00 MiB/s, done.
From https://github.com/pyenv/pyenv
   bf193666..e13b5848  master     -> origin/master
python-build 3.13.2 /home/PALANTHIR/arjan/.asdf/installs/python/3.13.2
Downloading Python-3.13.2.tar.xz...
-> https://www.python.org/ftp/python/3.13.2/Python-3.13.2.tar.xz
Installing Python-3.13.2...

BUILD FAILED (openSUSE 20250611 using python-build 2.6.2-2-ge13b5848)

Inspect or clean up the working tree at /tmp/python-build.20250616083639.305007
Results logged to /tmp/python-build.20250616083639.305007.log

Last 10 log lines:
./configure: line 5294: 5: Bad file descriptor
./configure: line 5294: 5: Bad file descriptor
./configure: line 5294: 5: Bad file descriptor
./configure: line 5294: 5: Bad file descriptor
checking whether the C compiler works... ./configure: line 5341: 5: Bad file descriptor
no
./configure: line 5386: 5: Bad file descriptor
configure: error: in `/tmp/python-build.20250616083639.305007/Python-3.13.2':
configure: error: C compiler cannot create executables
See `config.log' for more details
error installing version: failed to run install callback: exit status 1

You can see the command downloads the specific Python version, tries to compile it, and fails. Usually when a compile fails on Linux there is something wrong with your local environment, so I spent hours checking what was wrong with my local environment until I noticed that the command just worked perfectly when run as a local user on my laptop.
Upon investigating what exactly went wrong I started to notice the command had issues during the configure process:

configure:5317: checking whether the C compiler works
configure:5339: gcc  -I/home/PALANTHIR/arjan/.asdf/installs/python/3.13.2/include -L/home/PALANTHIR/arjan/.asdf/installs/python/3.13.2/lib -Wl,-rpath,/home/PALANTHIR/arjan/.asdf/installs/python/3.13.2/lib conftest.c -L/home/PALANTHIR/arjan/.asdf/installs/python/3.13.2/lib -Wl,-rpath,/home/PALANTHIR/arjan/.asdf/installs/python/3.13.2/lib >&5
configure:5343: $? = 1
configure:5383: result: no
configure: failed program was:
configure:5388: error: in `/tmp/python-build.20250616083639.305007/Python-3.13.2':
configure:5390: error: C compiler cannot create executables
See `config.log' for more details

the line 5294 as mentioned above in the error description on the configure script does this:

cat conftest.er1 >&5

Since the error "Bad file descriptor" usually means a file does not exist, it HAD to be the part of the command ">&5" which basically means sending the output to a specific file stream used by configure. This has nothing to do with a local environment. So I started searching on the internet if someone already had this problem. After hours of searching I finally found a hint: the problem arises when the user is a Samba user and indeed my Linux installs are joined in an ActiveDirectory domain using winbind, which is part of the Samba package.
And then I found a bug in Samba version 4.18. But I am using Samba 4.22 so the bug apparently is back.

I tried to test it. I changed one line in the /etc/nsswitch.conf file from:

passwd:     compat systemd winbind

passwd:     compat systemd

and then tried again, and it worked!

By this time it was 3 AM in the night. Since I do not have an account on the Samba Bugzilla site I could not leave a comment on this bug to re-open it. I have sent the Samba team an email to notify them of my find. I will update this page if I hear back from them.

For me, this is really an edge case of what can happen with software. Because it was so late in the night I did not bother to investigate what is causing this bug. It could however do it if I had the time. This is what I like about open source software.

Update 2025-06-16 14:04

The Samba team decided to give me a user account on their Bugzilla, so I updated the bug report there also samba-4.18.0 and above causes pyenv to fail with Bad file descriptor

Tracking history in database data

2025-05-27T00:00:00+02:00

This week I noticed strange behaviour in a system containing a table that is pretty straight forward and does not have any history logging, versioning or anything like that. The contents of the table are updated by automatic processes without logging also. Since two weeks there have been updates to the contents of this table which I did not notice in the years before. At such moments it is difficult to track where the updates come from or maybe who updated it manually. This situation always results in guessing what happened. In the end I restored the contents of the table from 2 weeks ago (thank you backups!) but this is generally not ideal.

History in database data in my opinion is about keeping track of changes of database records. For example, you want to keep track of when and by whom a database record was updated. This is the case when you do not want to lose earlier data or want to be able to see what the value was of a specific database record in the past. Most importantly, if a record is deleted from the database, you will never again know what the contents was of that specific database record.

This can be implemented for any database type and product, but there are differences how you can choose to implement them. In practice I see basically two variants of how to implement this. As I am mostly working with relational databases, let's take a moment to look at those variants:

Versioning table

Write a copy of a record to another table every time a modification on a record is executed. This is handled by code or database triggers that store a full copy of the database record in another table.
An example like this:

CREATE TABLE book (
  id INT,
  bookname VARCHAR(50),
  author VARCHAR(50),
  etc
);

CREATE TABLE book_version (
  id INT,
  time DATETIME,
  versioned_by VARCHAR(50),
  book_id INT,
  bookname VARCHAR(50),
  author VARCHAR(50),
  etc
);

Note, SQL commands in this article are not meant to directly execute on a database. They are solely for explanatory purposes to indicate the subject.

In this example we have a table book, and just before a mutation of a book is written to the database, a copy of the old record is stored in the book_version, along with the timestamp and name of the user that updated the book. A lot of Object - Relation Mapping (ORM) libraries for different languages support this kind of history keeping.
The usual history flow of a database record in the table book happens as follows: 1. When a new book is added to the database, an SQL INSERT command is used.
Some ORM libraries add also an initial version to the book_version table, but not all. 2. When a book is updated, just before issuing the SQL UPDATE command, this happens:

INSERT INTO book_version (time, versioned_by, book_id, bookname, author)
SELECT CURRENT_TIMESTAMP, 'username', id, bookname, author
FROM book
WHERE id = ?

Step 2 also happens when a book is removed from the book table before executing the DELETE command.

This way you can very easily get a full history of records in a database.

Advantages

Minimal overhead.
History is separated from the current version of the data.
Better normalization of data.

Disadvantages

The only way to detect if a record was deleted from the table book is to examine the book_history table. Only if the book_id is present in the book_history table and not in the book table, can you determine this record was present in the table at some moment.
While you can see the old values of the records in the book_history table, it is more difficult to compare the versioned records to the current state of the record. You have to write a join between the tables to select both current and past data. In short: the normalization is a disadvantage in this case.

Keeping track of modifications in the same table

Basically, this variant adds timestamp columns to the table for each record:

CREATE TABLE book (
  objectid BIGINT,
  book_id INT,
  bookname VARCHAR(50),
  author VARCHAR(50),
  etc,
  added_dt TIMESTAMP,
  added_by VARCHAR(50),
  modified_dt TIMESTAMP,
  modified_by VARCHAR(50),
  removal_dt TIMESTAMP,
  removal_by VARCHAR(50)
);

Now, every time a record is inserted, only the added_dt and added_by columns are filled (along with the content columns).
For updates to the record, two things happen:
1. Mark the old record as removed by setting the removal_dt and removal_by columns.
2. Insert a new record based on the existing record with the new values, and by also updating the modified_dt and modified_by columns.
Deleting a record also means updating the removal_dt and removal_by columns. This way, the most recent data of the records is always available by using a SELECT query like this:

SELECT *
FROM book
WHERE removal_dt IS NULL;

If you want to see how the changes of the book progressed over time, just write a query like this without the use of the removal_dt column in the WHERE clause:

SELECT *
FROM book
ORDER BY objectid;

which gives a result like this:

objectid	book_id	bookname	author	added_dt	added_by	modified_dt	modified_by	removal_dt	removal_by
1	1	The fellowship of the Ring	Tolkien	2025-05-27 08:00:00	Arjan van der Veen	NULL	NULL	2025-05-27 08:00:15	Arjan van der Veen
2	1	The fellowship of the Ring	J.R.R. Tolkien	2025-05-27 08:00:00	Arjan van der Veen	2025-05-27 08:00:15	Arjan van der Veen	2025-05-27 08:00:30	Arjan van der Veen
3	1	The Fellowship of the Ring	J.R.R. Tolkien	2025-05-27 08:00:00	Arjan van der Veen	2025-05-27 08:00:30	Arjan van der Veen	NULL	NULL

Here I added "The Fellowship of the Ring" to the table, and updated it a few seconds later by correcting the name of the author and then the title.

Advantages

Minimal overhead.
History is stored within the same table, so it is easier to see what changed over time.
Deleted records are still visible for a DBA.
The latest change is always visible with the current record (in this case record with objectid 3).

Disadvantages

You have to extend WHERE clauses of your SQL statements everywhere to use the removal_dt IS NULL to get your current data. This can easily be solved by using a database view for this table. But since almost all SQL queries have a WHERE clause this is no issue for me personally.
More complex to use this database design with an ORM (Object - Relation Mapping) library when coding.

In the past I wrote an extension to Java Hibernate ORM to automatically make use of this pattern for a client, but I do not have the license to share that code. It is doable though to implement it and significantly makes coding an application using this easier. Nowadays I just write SQL commands directly and noticed that, for me personally, it does not take much longer to develop applications with this database design.

Vernieuwde website

2025-03-04T15:20:00+01:00

Na een algemene website gehad te hebben met eigenlijk alleen een CV er op is het tijd voor een vernieuwing. Meer dan 20 jaar geleden was de site ongewijzigd en de tijden zijn veranderd. Op deze vernieuwde site kan je nog steeds mijn CV vinden als oprichter van Palanthir, maar ook artikelen over technische IT zaken, of meer om kennis te delen over uiteenlopende zaken.

Scaling KDE apps on Gnome Wayland

2025-03-01T16:30:00+01:00

When using Gnome on Linux with Wayland in combination with font scaling using Gnome Tweaks, KDE apps are not scaled accordingly.
One solution is to set an environment variable in ~/.config/environment.d/kdeapps.conf containing the scale factor for KDE apps like this:

QT_SCALE_FACTOR=1.3

Reboot, and after that the KDE apps are scaled accordingly.

IPv6 NAT on libvirt virtual machine

2023-12-15T00:00:00+01:00

librtvirt supports IPv6 for virtual machines, but that is usually a bridge solution or a subnet of the /64 the libvirt host is connected to. But I'm in the situation working on a network without IPv6, so I get my IPv6 address from a Wireguard interface. To get IPv6 working in a virtual machine on my host, I do not have a range available on my host machine without extensive changes to the wireguard VPN.

The solution is to use NAT with IPv6.

First, create a new libvirt NAT network with IPv6 enabled and the ranges you want. For the ipv6 range I chose fd00::/64 because that is a private range like 10.x.x.x.

<network connections="1">
  <name>virbr1</name>
  <uuid>a592c6e0-b9ed-463a-a0a3-3feef8c01b7d</uuid>
  <forward mode="nat">
    <nat>
      <port start="1024" end="65535"/>
    </nat>
  </forward>
  <bridge name="virbr1" stp="on" delay="0"/>
  <mac address="52:54:00:74:c6:cf"/>
  <domain name="virbr1"/>
  <ip address="192.168.100.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.100.128" end="192.168.100.254"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="fd00::1" prefix="64">
    <dhcp>
      <range start="fd00::100" end="fd00::1ff"/>
    </dhcp>
  </ip>
</network>

Because we want to do nat and I could not find a libvirt network filter rule for doing nat, I used the hook mechanism of libvirt, which means putting a script called network in /etc/libvirt/hooks:

#!/bin/bash
VIRT_NET_MATCH="virbr1"
VIRT_BRIDGE_MATCH="virbr1"
IPV6_DEFAULT_INTERFACE="vpn-thuis"
if [ "$1" == "$VIRT_NET_MATCH" ]; then
  VIRT_BRIDGE=$VIRT_BRIDGE_MATCH
  if [ "$2" == "started" ]; then
    echo "Adding NAT MASQUERADING entries for $VIRT_BRIDGE"
    ip6tables -t nat -A POSTROUTING -s fd00::/64 -o $IPV6_DEFAULT_INTERFACE -j MASQUERADE
  elif [ "$2" == "stopped" ]; then
    echo "Stopping $1 ($VIRT_BRIDGE) ..."
    ip6tables -t nat -D POSTROUTING -s fd00::/64 -o $IPV6_DEFAULT_INTERFACE -j MASQUERADE
  fi
fi

Make sure this file is executable. IPV6_DEFAULT_INTERFACE is the mail IPv6 interface over which the natting should take place. The rest is self explanatory I think. When libvirt brings the network interface online, it calls the /etc/libvirt/hooks/network script, and this handled the creation or deleten of the NAT rule in the firewall of the host.